IMPROVED AUTOMATIC LIP-READING BASED ON THE EVALUATION OF INTENSITY LEVEL OF SPEAKER’S EMOTION

Abstract. Automatic audio-visual speech recognition systems (AVSRs) have recently achieved tremendous success, especially in limited vocabulary tasks by far surpassing human abilities to recognize speech, acoustically noisy conditions. based on processing of audio and video information are being actively researched developed all over the world. However, scientific studies aimed at analyzing influence speaker's emotional state (anger, disgust, fear, happy, neutral, sad), and, most importantly, intensity level emotion (low - LO, medium MD, high HI) automatic lip-reading not been conducted. In this regard, relevance research topic cannot be overestimated requires detailed study. paper, we present a novel approach for lip-reading, that includes evaluation speaker’s its level. The proposed uses visual data detect person’s type assigns it one trained models. This essentially resolves multi-emotional issue associated with real-life scenarios. improves state-of-the-art results due consideration pronounced up 8.2% terms accuracy. Current is first step creation emotion-robust leaves open wide field further research.